The sustained popularity of mobile handheld devices has contributed to a proliferation of wireless networks and hotspots for accessing and viewing online multimedia content. This has led, in turn, to development of technology for wirelessly streaming content such as online videos, movies, games and web pages from mobile devices to external high definition displays.
Several peer-to-peer wireless technologies have been developed for securely and wirelessly sending video and audio from a source device to a remote display device. Examples of proprietary technologies include WiDi developed by Intel Corporation, and AirPlay developed by Apple Inc. In an effort to provide an open standard, the Wi-Fi Alliance (WFA) has developed the Miracast™ standard, which uses Wi-Fi Direct™ interconnect supported devices without the need for a wireless access point.
According to the Miracast™ specification, when two devices connect with each other directly, one assumes the role of the source (transmitting device) and the other becomes a sink (the device receiving and rendering the content to the user). In operation, a real-time digital video screen capture from the source device is streamed via a WLAN (wireless local area network) for display via the sink device. Since the video content in the display buffer of the source device is in raw format (e.g. YUV colour space), the content must be compressed by an encoder (e.g. x264 encoder), taking into account the display device capabilities and network conditions in order to optimize transmission over the Wi-Fi interface. The display device (sink) receives the content, decodes it, and then displays the content. However, the compression introduced by encoding comes at the expense of image quality and/or latency. In order to maintain a satisfactory end-user experience, a minimum level of video quality must be provided. This requires adjusting the compression and latency parameters of the encoder. For example, in an x264 encoder the output bit rate can be constrained using VBV (video buffer verifier) encoding, and encoder latency can be adjusted by varying the number of frames held by the x264 internal buffer. The required level of compression is dictated by the type and quality of the WLAN link, including supported rates and network conditions. For example, a 2×2 spatial multiplexing WLAN link provides an over-the-air (OTA) throughput of 130 Mbps whereas a 1×1 WLAN link provides only 65 Mbps. The latency requirement of the encoder is an integrant of the end-to-end latency (or glass-to-glass latency, which captures the time elapsed between the first pixel appearing on the source device and appearing on the sink device). The latency requirement dictates the class of supported applications over a given link. For example, a 100 msec end-to-end latency budget may be sufficient to support a web browsing application whereas heavy interactive gaming may require a less than 50 msec end-to-end budget.
A significant portion of the end-to-end latency budget may be consumed by the input lag of the sink device, which is defined as the difference between the time that a signal is input into the sink device and the time that it is shown by the display. This input lag time has been measured as high as 68 ms or the equivalent of 3-4 frames on a 60 Hz display. The greater the input lag of the sink device, the less compression that can be provided to the signal. Since the input lag is often unknown, it is customary to assume a conservative input lag contribution to the end-to-end latency when choosing the compression level during source encoding.
The WFA maintains a current list of Miracast™ Certified devices for which the input lag is known. Therefore, in some implementations the input lag is stored as part of the extended display identification data message (display_edid), and can be used in calculating the level of compression to be applied at the source device for a particular application. However, there are many implementations where this information is not known.
It is therefore desirable to provide a solution to the problem of estimating input lag in applications where the input lag is not known.